Vision-Based Reinforcement Learning for RoboCup : Towards Real Robot Competition

We have been doing a research on vision-based reinforcement learning and applied the method to build real soccer playing robots towards RoboCup Initiative. In the first stage [Asada et al., 1994a; Asada et al., 1995b], a robot learned to shoot a ball into a goal given the state space in terms of the sizes and the positions of both the ball and the goal in image. As one extension of this first achievement, the method for autonomous state space construction is proposed [Asada et al., 1995a]. In the second stage [Asada et al., 1994b; Asada et al., 1994c], we set up an opponent just before the goal, that is, a goal keeper, and make the robot learn to shoot a ball into a goal avoiding the goal keeper. The behavior of the opponent is scheduled for the learner to efficiently obtain the desired behavior [Asada et al., 1995c]. This paper describes several research issues for RoboCup with real robots along with our research projects.